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THE VALIDITY OF THE WOLFF AND STEIN CONCLUSIONS (SIX 
MONTHS LATER. STUDY I. PS ODD 281) IS CHALLENGED ON THE BASIS 
OF ONE MAJOR AND FOUR MINOR METHODOLOGICAL DEFICIENCIES. THE 
STUDY'S MAJOR CONCLUSION WAS THAT FORMER HEAD START CHILDREN 
HAVE GREATER LEARNING READINESS THAN THEIR CLASSMATES HAVE 
SIX MONTHS LATER BUT THAT NO EDUCATIONAL GAINS HAD BEEN MADE. 
THE MAJOR CRITICISM IS THAT, ALTHOUGH ECONOMIC AND SOCIAL 
ADVANTAGES GREATLY FAVORED NON-HEAD START CHILDREN, WOLFF AND 
STEIN CONSIDERED THE ADVANTAGE TO BE "SLIGHT." THE 
ALTERNATIVE TO WOLFF'S CONCLUSION IS THAT CHILDREN FROM 
DEPRIVED HOMES ARE ABLE TO HOLD THEIR OWN WITH CLASSMATES 
FROM BETTER ADVANTAGED FAMILIES AS A RESULT OF HEAD START 
ENRICHMENT. (LG) 
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MEMO: Comments on the Wolff and Stein Study 

This study compares the performance of 179 children who had had Head 
Start with 388 of their classmates who had not. The children were 
enrolled in 40 kindergarten classes, taught by 15 different teachers 
•• in four different schools; two of them Puerto Rican, one Negro, and 
one mixed , . ‘ 

The major conclusion of the study, as dra^m by the authors, is twor- • 
fold (p, 83): 

The findings of this study show that overall,- the children- who had 
Head Start still have greater readiness for learning than their 
classmates, six months later. * ' . 

They also show that in the kindergarten studied, no educational gains 
had been made despite their greater .’’thirst for knowledge.” 

The conclusion concerning the lack of educational gain is based on 
the absence of significant differences between Head Start children and 
their controls, • . 

The validity of the conclusions of this study clearly depends on the 
comparability between the Head Start' and control groups at the point 
of entry into kindergarten. On this score, the authors conclude (p, 8) 

“These findings indicate clearly that the home environnients of both 
the Head Start children and the non-Head Start children in the kinder- 
garten studies are very similar with' a slight economic and social 
advantage in favor of the non-Head Start children in each of the* 
ethnic groups. * 

For that reason the changes that the study found in the two groups of 
children after six to eight months of kindergarten can be attributed 
primarily to school centered rather than to any home centered factors." 

In my judgment, the assumption of similarity mads by the authors can 
be seriously called to question, for, on examination, the economic and 
social advantages favoring the non-Head Start children can hardly be 
dismissed as "slight”. They include the following (data taken from Tables 
1 and 2 of the study): 1) the percentage of families in the Head Start 

group with incomes of $5000/year or better is 27; the corresponding 
percentage for the control group is 41. Consistent with this fact, the 
number of Head Start families on welfare or ADC is twice that for the 
control group, 2) Among the Head Start families the percentage with 
more than three children in the home is 44. (the corresponding figure 
for the controls is 29.) Given the well-established negative correlation 
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between family size and intellectual level, particularly in lower class 
poups, the higher number of large families in the Head Start group can 
hardly be disregarded. 3) With respect to amount of schooling, the 
proportion of fathers of Head Start pupils with an education of eighth 
grade or le«s was 38; the corresponding figure for the controls wfis 24. 
Although the comparable difference was negligible for mothers, .37% of 
the- Head Start mothers had high school education or better; the 
corresponding figure for the .control group was 50%* 

The conclusion that the Head Start made no educational gains is based 
on the absence of any significant difference in score obtained by 
, experimental and controlled groups after approximately 6 months 
attendance in kindergarten. No measure of educational status was 
obtained at the beginning of kindergarten, or for that matter, at the 
time that the exprimental group entered Head Start. The investigators 
apparently assuming that if the experimental group got any "head start" 
at all from their pre-school experience, the control group was able to 
catch up to them by the sixth month of kindergarten. Such an inference 
of course rests on the assumption that both experimental and control 
groups were at comparable levels of intellectual development before the 
- Head Start program began. No evidence is provided in support of this 

assumption. On the contrary, as we have seen, the data cited by the 

authors poiht strongly to the probability that the control group had a ' 

higher level of intellectual development than the experimental group 
before the. Head Start experience was introduced. This . conclusion 
follows from the well established correlations existing between measures 
of intellectual performance and such variables as family income and 
family size. 
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In the light of the foregoing considerations, the fact that the Head 
Start and non-Head Start groups show comparable. scores after six months 
of kindergarten represents . testimony favoring the effectiveness of the 
Head Start experience rather than the reverse. In other words, what the • 
data see*..s to show is that children coming from relatively deprived 
families who have a Head Start program are able to hold their om with 
children from better advantaged families well into the first year of 
school. When one adds to this fact the additional finding of the study 
which seems to be well established, that the Head. Start children have a’ 
greater readiness for learning than their classmates, the impact of the 
program takes on even greater significance, . . 



In ad’dition to the major flow outlined above, the study suffers from a 
number of additional methodelogical deficiencies; 



1 ) Ordinarily one makes analyses in terms of total score since this is 
the most reliable and best index of the child*s status. When such a 
score is computed from the data provided in Table 8 (the tests of 
significance employed by the authors, all of them yielding a negative 
result, were made separately for each of the part scores of* the Pre- 

Schoox Inventory instrument), the results indicate an average that is 
10 . points higher for the Head Start group than for the controls. 















Altliough the data cited do not permit a test of statistical significance, 
there is a good possibility that the Head Start group actually scored 
reliably higher than the controls. 

2) In their test of significance the authors employ error scores based 
on Individual differences among subjects without regard to school or 
classroom. Since, in point of fact, the children x^^ero grouped in 30 
different classrooms in 4 different schools, there is the possibility 
that there were significant differences from classroom to classroom. 

This factor is not taken into account in the statistical analysis: that 

is, an appropriate design x^^ould have shox^n individuals as nested in 
classrooms and classrooms as nested in schools, x^^ith appropriate' 
partitioning of variance into classroom and school components. 

'I . * 

3 ) An even more important omission is the failure to examine separately 
scores for boys and girls. Previous studies have shovm appreciable 
differences in scores for the tx^^o sexes at this age level, particularly 
in the case of Kegro children where males have shox-m lox^^er scores, but 
greater gains attributable to pre-schoql programs. 

4) The authors state that "to prevent any bias for or against Head Start 
from influencing the rank given a child," the teachers were not told 
that this .study Vas being done in connection with Head Start evaluation. 
Although such precaution is desirable, the critical question still 
remains as to xv’hether the teachers knex-/ V/he'ther a given child had or had 
not a Head Start program. Given such knox^^ledge, the teacher *s evaluation 
of the child cannot be assumed to be free of possible bias for or against 
’the Head Start experience. 

Both the major and minor deficiencies outlined above reflect a general 
weakness in methodology characterizing the study as a whole. Many of 
the statistical procedures employed are crude if not wholly inappropriate. 
It is doubtful whether in its present form the study xrould have been 
accepte ' for publication in any established scientific journal. 



P.S. Incidentally, the study x^ith the best matched control group that 
I have run across to date is the one conducted by one of my o\m graduate 
students, Robert R. Rice, nox<^ at the University of Missouri x^^here he 
worked with a Head Start group in Kansas City. At the end of approximately 
eight months of kindergarten his Head Start group of 104 Negro children 
had- a mean score of approximately 18 points higher on the CaldX'/ell Pre- 
School Inventory than a control group of Negro children from the same 
housing project. Since Rice used the same instrument employed in the 
Wolff study, his results give us some indication of the kinds of results 
that arc obtained when the control group is closely matched to , the 
experimental group at the outset o‘f the study. 



